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Abstract 

Moment closure methods appear in myriad scientific disciplines in the mod¬ 
elling of complex systems. The goal is to achieve a closed form of a large, usually 
even infinite, set of coupled differential (or difference) equations. Each equation 
describes the evolution of one “moment”, a suitable coarse-grained quantity com¬ 
putable from the full state space. If the system is too large for analytical and/or 
numerical methods, then one aims to reduce it by finding a moment closure re¬ 
lation expressing “higher-order moments” in terms of “lower-order moments”. In 
this brief review, we focus on highlighting how moment closure methods occur in 
different contexts. We also conjecture via a geometric explanation why it has been 
difficult to rigorously justify many moment closure approximations although they 
work very well in practice. 


1 Introduction 

The idea of moment-based methods is most easily explained in the context of stochastic 
dynamical systems. Abstractly, such a system generates a time-indexed sequence of 
random variables x = x(t) G X , say for t G [0, +oo) on a given state space X. Let 
us assume that the random variable x has a well-defined probability density function 
(PDF) p = p(x,t). Instead of trying to study the full PDF, it is a natural step to just 
focus on certain moments m ; = m, (t) such as the mean, the variance, and so on, where 
j G J and J is an index set and M = {nij : j G J} is a fixed finite-dimensional space 
of moments. In principle, we may consider any moment space M consisting of a choice 
of coarse-grained variables approximating the full system, not just statistical moments. 
A typical moment-closure based study consists of four main steps: 

(50) Moment Space: Select the space M containing a hierarchy of moments nij. 

(51) Moment Equations: The next step is to derive evolution equations for the mo¬ 
ments nij. In the general case, such a system will be high-dimensional and fully 
coupled. 
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(52) Moment Closure: The large, often even infinite-dimensional, system of moment 
equations has to be closed to make it tractable for analytical and numerical tech¬ 
niques. In the general case, the closed system will be nonlinear and it will only 
approximate the full system of all moments. 

(53) Justification &: Verification: One has to justify, why the expansion made in 
step (SI) and the approximation made in step (S2) are useful in the context of 
the problem considered. In particular, the choice of the rrij and the approximation 
properties of the closure have to be answered. 

Each of the steps (S0)-(S3) has its own difficulties. We shall not focus on (SO) as 
selecting what good ’moments’ or ’coarse-grained’ variables are creates its own set of 
problems. Instead, we consider some classical choices. (SI) is frequently a lengthy com¬ 
putation. Deriving relatively small moment systems tends to be a manageable task. 
For larger systems, computer algebra packages may help to carry out some of the cal¬ 
culations. Finding a good closure in (S2) is very difficult. Different approaches have 
shown to be successful. The ideas frequently include heuristics, empirical/numerical ob¬ 
servations, physical first-principle considerations or a-priori assumptions. This partially 
explains, why mathematically rigorous justifications in (S3) are relatively rare and usu¬ 
ally work for specific systems only. However, comparisons with numerical simulations of 
particle/agent-based models and comparisons with explicit special solutions have consis¬ 
tently shown that moment closure methods are an efficient tool. Here we shall also not 
consider (S3) in detail and refer the reader to suitable case studies in the literature. 

Although moment closure ideas appear virtually across all quantitative scientific dis¬ 
ciplines, a unifying theory has not emerged yet. In this review, several lines of research 
will be highlighted. Frequently the focus of moment closure research is to optimize clo¬ 
sure methods with one particular application in mind. It is the hope that highlighting 
common principles will eventually lead to a better global understanding of the area. 

In Section [2] we introduce moment equations more formally. We show how to derive 
moment equations via three fundamental approaches. In Section [3] the basic ideas for 
moment closure methods are outlined. The differences and similarities between different 
closure ideas are discussed. In Section [4] a survey of different applications is given. As 
already emphasized in the title of this review, we do not aim to be exhaustive here but 
rather try to indicate the common ideas across the enormous breadth of the area. 
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2 Moment Equations 

The derivation of moment equations will be explained in the context of three classical 
examples. Although the examples look quite different at first sight, we shall indicate 
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how the procedures are related. 


2.1 Stochastic Differential Equations 

Consider a probability space (12, F, P) and let W = W ( t ) S 1 L be a vector of independent 
Brownian motions for tgR. A system of stochastic differential equations (SDEs) driven 
by W{t) for unknowns x = x(t) £ = X is given by 

da; = /( x) df + F(x) dW (1) 

where / : R N R^, F : R N ->• R NxL are assumed to be sufficiently smooth maps, and 
we interpret the SDEs in the Ito sense [M2]. Alternatively, one may write ([1]) using 
white noise, i.e., via the generalized derivative of Brownian motion, £ := W' [3 as 

x' = f{x) + F{x% ' = A ( 2 ) 

For the equivalent Stratonovich formulation see 03 • Instead of studying m-m directly, 
one frequently focuses on certain moments of the distribution. For example, one may 
make the choice to consider 


mj(t) := (a ;(f) J ) = {xi (t) n ■ ■ ■ x N (t) JN ), (3) 

where (•) denotes the expected (or mean) value and j £ J, j = (ji, ■ • •, jiv), jn £ No, 
where J is a certain set of multi-indices so that M = {rrij : j £ J}. Of course, it 
should be noted that J can be potentially a very large set, e.g., for the cardinality of all 
multi-indices up to order J we have 

j G til = E ^<J 

n 

However, the main steps to derive evolution equations for m- } are similar for every fixed 
choice of J,N. After defining toj = m$(t) (or any other “coarse-grained” variables), 
we may just differentiate mj. Consider as an example the case N = 1 = L, and J — 
{1,2 ,...,</}, where we write the multi-index simply as j = j £ No- Then averaging (|2|) 
yields 

m' 1 = (x') = (/(x)) + {F(x)g), (5) 

which illustrates the problem that we may never hope to express the moment equations 
explicitly for any nonlinear SDE if / and/or F are not expressible as convergent power 
series, i.e., if they are not analytic. The term (F(x)£) is not necessarily equal to zero for 
general nonlinearities F as f ( * F(x(s)) dlT(s) is only a local martingale under relatively 
mild assumptions m • Suppose we simplify the situation drastically by assuming a 
quadratic polynomial / and constant additive noise 

f(x) = a 2 X 2 +aiX + ao, F(x) = cr£R. (6) 

Then we can actually use that (£} = 0 and get 

m'x = ( x') = a 2 (x 2 ) + ai(x) + a 0 = a 2 m 2 + aimi + a 0 . (7) 


J + N \ _(J + N)l 
J ) ~ J\N\ ' 


(4) 
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Hence, we also need an equation for the moment m 2 . Using Ito’s formula one finds the 
differential 

d(x 2 ) = [2 x/(x) + cr 2 ] df + 2xa AW (8) 

and taking the expectation it follows that 

m' 2 = 2(a 2 x 3 + a\x 2 + aox) + cr 2 + a(2x£) 

= 2(a2m 3 + aim 2 + a 0 mi) + cr 2 , (9) 

where (2x£) = 0 due to the martingale property of f* 2 x(s) AW S . The key point is that 
the ODE for ?n 2 depends upon m 3 . The same problem repeats for higher moments and 
we get an infinite system of ODEs, even for the simplified case considered here. For a 
generic nonlinear SDE, the moment system is a fully-coupled infinite-dimensional system 
of ODEs. Equations at a given order |j| = J depend upon higher-order moments |j| > J, 
where |j| := 

Another option to derive moment equations is to consider the Fokker-Plank (or for¬ 
ward Kolmogorov) equation associated to CD)-©; see HU- It describes the probability 
density p = p(x , t\xo, to) of x at time t starting at £0 = x(to) and is given by 


dp 

m 


E 


d 

dx k 


1 N 
\pf ] + 2 E 

i,k= 1 


d 2 

dxidxk 


[(FF T ) ik p\. 


( 10 ) 


Consider the case of additive noise F(x) = a , quadratic polynomial nonlinearity f(x) 
and N = 1 = L as in (O, then we have 


dp 

dt 


d 2 . , cr 2 d 2 p 

-a::[( a * x +aiX + a 0 )p) + — w-^. 


dx 


2 dx 2 


( 11 ) 


The idea to derive equations for rrij is to multiply m by , integrate by parts and use 
some a-priori known properties or assumptions about p. For example, we have 


m 


/ 

1 



dp 

x— da; 
dt 


/ — x-^-[(a, 2 X 2 + a\X + a 0 )p\ dx + 
Jr dx 


a 2 d 2 p 


dx. 


If p and its derivative vanish at infinity, which is quite reasonable for many densities, 
then integration by parts gives 


m i = / [(a 2 x 2 + ciix - 

/ TO) 


( 12 ) 


as expected. A similar calculation yields the equations for other moments. Using the 
forward Kolmogorov equation generalizes in a relatively straightforward way to other 
Markov process, e.g., to discrete-time and/or discrete-space stochastic processes; in fact, 
many discrete stochastic processes have natural ODE limits [51 l?7lf551f50j . In the context 
of Markov processes, yet another approach is to utilize the moment generating function 
or Laplace transform s 1 —>• (exp[isx]) (where i := y/— 1) to determine equations for the 
moments. 
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2.2 Kinetic Equations 

A different context where moment methods are used frequently is kinetic theory [3311531 
03] . Let x € S3 C and consider the description of a gas via a single-particle density 
g = g(x,t.,v), which is nonnegative and can be interpreted as a probability density if 
it is normalized; in fact, the notational similarity between p from Section 12.11 and the 
one-particle density g is deliberate. The pair (x, v) £ fl x is interpreted as position 
and velocity. A kinetic equation is given by 

^ + v-V x g = Q(g), (13) 

where \7 X = (g§^-, ■ • ■, suitable boundary conditions are assumed, and g H > Q(g) 

is the collision operator acting only on the ^-variable at each (x, t) £ l iV x [0, +oo) with 
domain V(Q). For example, for short-range interaction and hard-sphere collisions [105] 
one would take for a function v H > G(v) the operator 

Q(G)(v) = [ [ \\v — w\\[G(w*)G(v*) — G(v)G(w)] dw dip (14) 

Js N ~ l Jr n 

where v* = ^(v + tv + ||u — w\\ip), w* = ^(v + w + ||u — w\\ip) for ip £ and 

denotes the unit sphere in R w . We denote velocity averaging by 

(G) = [ G{v) dv, (15) 

Jrn 

where the overloaded notation (•) is again deliberately chosen to highlight the similarities 
with Section [2.II It is standard to make several assumptions about the collision operator 
such as the conservation of mass, momentum, energy as well as local entropy dissipation 

<Q(G)>=0, (vQ(G))= 0, <|M| 2 Q(G)) = 0, (ln(G)Q(G)} < 0. (16) 

Moreover, one usually assumes that the steady states of m are Maxwellian (Gaussian- 
like) densities of the form 

P*{v) = ( 2 tt 9 ) n / 2 6XP (~ ^ 20* ^ V *) £ R+ X R+ X RN ( 17 ) 

and that Q commutes with certain group actions (531 implying symmetries. Note that the 
physical constraints m have important consequences, e.g., entropy dissipation implies 
the local dissipation law 

^{glng- g) + V x - (v(glng- g)) = (In gQ(g)) < 0. (18) 

while mass conservation implies the local conservation law 

^te) + v x >e) = o (19) 

with similar local conservation laws for momentum and energy. The local conservation 
law indicates that it could be natural, similar to the SDE case above, to multiply the 
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kinetic equation m by polynomials and then average. Let {rrij = mj(v)}j =1 be a basis 
for a J-dimensional space of polynomials M. Consider a column vector M = M(v) £ 
containing all the basis elements so that every element m £ M can be written as m = 
a T M for some vector a £ R J . Then it follows 

a 

-(qM)+V x -(vqM) = (Q(q)M) (20) 

by multiplying and averaging. This is exactly the same procedure as for the forward 
Kolmogorov equation for the SDE case above. Observe that (l20l) is a J-dimensional 
set of moment equations when viewed component-wise. This set is usually not closed. 
We already see by looking at the case M = v that the second term in (121)1) will usually 
generate higher-order moments. 


2.3 Networks 


Another common situation where moment equations appear are network dynamical sys¬ 
tems. Typical examples occur in epidemiology, chemical reaction networks and socio¬ 
economic models. Here we illustrate the moment equations i 75111231 F156(1I4T | for the 
classical susceptible-infected-susceptible (SIS) model [32] on a fixed network; for re¬ 
marks on adaptive networks see Section [4] Given a graph of K nodes, each node can be 
in two states, infected I or susceptible S. Along an ST-link infections occur at rate r 
and recovery of infected nodes occurs at rate 7 . The entire (microscopic) description of 
the system is then given by all potential configurations x £ R.^ = X of non-isomorphic 
graph configurations of S and / nodes. Even for small graphs, N can be extremely 
large since already just all possible node configurations without considering the topology 
of the graph are 2 K . Therefore, it is natural to consider a coarse-grained description. 
Let mi = (I) = (I)(t ) and ms = (S) = (S)(t ) denote the average number of infected 
and susceptibles at time t. From the assumptions about infection and recovery rates we 
formally derive 


d ms 
d t 

dm/ 
d t 


7 mi - t(SI), 
t(SI) - 7m/, 


( 21 ) 

( 22 ) 


where (SI) =: msi denotes the average number of ST-links. In (12TT) the first term de¬ 
scribes that susceptibles are gained proportional to the number of infected times the 
recovery rate 7 . The second term describes that infections are expected to occur propor¬ 
tional to the number of ST-links at the infection rate r. Equation (1221) can be motivated 
similarly. However, the system is not closed and we need an equation for (SI). In ad¬ 
dition to (l2ll) - (l22l) . the result [1471 Thm.l] states that the remaining second-order motif 
equations are given by 


d m S i 
d t 

Amu 

At 

d m S s 
At 


7( m n ~ m S i ) + r(mssi ~ misi - m S i ), 
-2y mu + 2t(to/ S / + m S i), 

27 m S i - 2 rmssi, 


(23) 

(24) 

(25) 
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where we refer also to [75TT251 : it should be noted that (l251) - (E5l) does not seem to coincide 
with a direct derivation by counting links (33) (9.2)-(9.3)]. In any case, it is clear that 
third-order motifs must appear, e.g., if we just look at the motif IS I then an infection 
event generates two new II -links so the higher-order topological motif structure does 
have an influence on lower-order densities. If we pick the second-order space of moments 


M = {mi, m s , m S i, m S s, m n } 


(26) 


the equations m-m and (I23l) - (l25l) are not closed. We have the same problems as 
for the SDE and kinetic cases discussed previously. The derivation of the SIS moment 
equations can be based upon formal microscopic balance considerations. Another option 
is write the discrete finite-size SIS-model as a Markov chain with Kolmogorov equation 



(27) 


which can be viewed as an ODE of 2 K equations given by a matrix P. One defines the 
moments as averages, e.g., taking 

K K 

(I)(t) :=J2 kxik) ^’ < 5 >(i) :^(A'-^(f), (28) 

k =0 k =0 


where x^ k \t) are all states with k infected nodes at time t. Similarly one can define 
higher moments, multiply the Kolmogorov equation by suitable terms, sum the equation 
as an analogy to the integration presented in Section 12.21 and derive the moment equa¬ 
tions m- For any general network dynamical systems, moment equations can usually 
be derived. However, the choice which moment (or coarse-grained) variables to consider 
is far from trivial as discussed in Section [U 


3 Moment Closure 

We have seen that moment equations, albeit being very intuitive, do suffer from the 
drawback that the number of moment equations tends to grow rapidly and the exact 
moment system tends to form an infinite-dimensional system given by 

^ = h 1 (m 1 ,m 2 ,...), 

= h 2 (m 2 ,m 3 ,...), (29) 

dm 3 _ 

HT ~ 

where we are going to assume from now on the even more general case hj = hj ( m\ , m 2 , m 3 , 
for all j. In some cases, working with an infinite-dimensional system of moments may 
already be preferable to the original problem. We do not discuss this direction further 
and instead try to close (E51) to obtain a finite-dimensional system. The idea is to find a 
mapping II, usually expressing the higher-order moments in terms of certain lower-order 
moments of the form 

H(mi ,... ,m K ) = (m K+ i,m K+2 ,...) (30) 
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for some n £ J, such that (EH1) yields a closed system 


dmi 

~ar 

= hi (mi, m2,.. 

. ,m K ,H(mi,.. 



Clra 2 

= h 2 (mi,m 2 ,.. 

■ ,m K ,H(mi,.. 


( 31 ) 

dm K 
d t 

= h K (mi,m 2 ,.. 

■, m K ,H(mi,.. 

■ j m K )). 



The two main questions are 

(Ql) How to find/select the mapping HI 

(Q2) How well does (I3T1) approximate solutions of (1291) and/or of the original dynamical 
system from which the moment equations (1291) have been derived? 

Here we shall focus on describing the several answers proposed to (Ql). For a gen¬ 
eral nonlinear system, (Q2) is extremely difficult and Section 13.41 provides a geometric 
conjecture why this could be the case. 

3.1 Stochastic Closures 

In this section we focus on the SDE CD from Section 12.11 However, similar principles 
apply to all incarnations of the moment equations we have discussed. One possibility is 
to truncate [140| the system and neglect all moments higher than a certain order, which 
means taking 

= ( 0,0,...). (32) 

Albeit being rather simple, the advantage of (l32l) is that it is trivial to implement and 
does not work as badly as one may think at first sight for many examples. A variation 
of the theme is to use the method of steady-state of moments by setting 

0 = h K+ i(mi,m 2 ,... ,m K ,m K+ i,...), 

0 = h K+ 2 (mi, m 2 ,. -., m K , m K +i,...), ( 33 ') 


and try to solve for all higher-order moments in terms of (mi, m 2 ,..., m K ) in the algebraic 
equations (l33l) . As we shall point out in Section [3741 this is nothing but the quasi-steady- 
state assumption in disguise. Similar ideas as for zero and steady-sate moments can also 
be implemented using central moments and cumulants |140l . 

Another common idea for moment closure principles is to make an a priori assump¬ 
tion about the distribution of the solution. Consider the one-dimensional SDE example 
( N = 1 = L) and suppose x = x(t) is normally distributed. For a normal distribution 
with mean zero and variance v 2 , we know the moments 

( x J ) = v^(j — 1)!!, if j is even, ( x = 0, if j is odd, 

so one closure method, the so-called Gaussian (or normal) closure, is to set 

nij = 0 if j > 3 and j is odd, 
nij = (j — 1)!! if j > 4 and j is even. 


( 34 ) 






A similar approach can be implemented using central moments. If x turns out to deviate 
substantially from a Gaussian distribution, then one has to question whether a Gaussian 
closure is really a good choice. The Gaussian closure principle is one choice of a wide 
variety of distributional closures. For example, one could assume the moments of a 
lognormal distribution [35] instead 

x ~ exp[/I + vx\, x ~ Af(0, 1), => (x J ) = rrij = exp 

where means ’distributed according to’ a given distribution and 7V(0,1) indicates the 
standard normal distribution. Solving for (p,v) in (1351) in terms of (mi,m2) yields a 
moment closure ( 7713 , 7714 , ■ • ■) = H ( 7711 , 7712 ). The same principle also works for discrete 
state space stochastic process, using a-prior distribution assumption. A typical example 
is the binomial closure [BUI and mixtures of different distributional closure have also been 
considered [841185] . 

3.2 Physical Principle Closures 

In the context of moment equations of the form (1201) derived from kinetic equations, 
a typical moment closure technique is to consider a constrained closure based upon 
a postulated physical principle. The constraints are usually derived from the original 
kinetic equation (I13f) . e.g., if it satisfies certain symmetries, entropy dissipation and local 
conservation laws, then the closure for the moment equations should aim to capture these 
properties somehow. For example, the assumption 

span{l, 7)i,, v N , ||u|| 2 } C M (36) 

turns out to be necessary to recover conservation laws [93] , while assuming that the space 
M is invariant under suitable transformations is going to preserve symmetries. However, 
even by restricting the space of moments to preserve certain physical assumptions, this 
usually does not constraint the moments enough to get a closure. Following [33] suppose 
that the single-particle density is given by 

g = 9Jt(a) = exp[a T Af(u)], m = m(v) £ M s.t. m(v) = a 1 M(v) (37) 

for some moment densities a = a(x,t ) £ R J . Using (1371) in (130l) leads to 

-<9Jt(a)M) + V, • (vm(a)M) = (Q(m(a))M). (38) 

Observe that we may view (1381) as a system of J equations for the J unknowns a. Hence, 
one has formally achieved closure. The question is what really motivates the exponential 
ansatz m ■ Introduce new variables 77 = ( l 0Jl(a)M) and define a function 

H(rj) = -(Ort(a)) + a T ?7 (39) 

and one may show that a = [D v H](rf). It turns out (93: that H(rj) can be computed by 
solving the entropy minimization problem 

min{(pln g — g) : (Mg) = 77 } = 17 ( 77 ), (40) 


■ ~ . 7 - 2-2 

dl 1 +2 J v 


(35) 
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where the constraint (Mg) = rj prescribes certain moments; we recall that M = M(v) 
is the fixed vector containing the moment space basis elements and the relation a = 
[D v H](if) holds. From a statistical physics perspective, it may be more natural to 
view (IdOD as an entropy maximization problem |70| by introducing another minus sign. 
Therefore, the choice of the exponential function in the ansatz E3 does not only guar¬ 
antee non-negativity but it was developed as it is the Legendre transform of the so-called 
entropy density p i-> g In g — g so it naturally relates to a physical optimization prob¬ 
lem [ 55] . 

To motivate further why using a closure motivated by entropy corresponds to certain 
physical principles, let us consider the ’minimal’ moment space 


M = span{ 1,i>i, ...,v N , |M| 2 } 


(41) 


The closure ansatz ED) can be facilitated using the vector M(v) = (1, v ±,..., vn, |M| 2 ) 
but then [53] the ansatz is related to the Maxwellian density (ED since 


p*(v) = exp[a T M(u)], a = (In 


q \ IKH u* 1 

(27T(9 ) 3 / 2 ) 26 ’ 6 ’ 26 


(42) 


but Maxwellian densities are essentially Gaussian-like densities and we again have a 
Gaussian closure. Using a Gaussian closure implies that the moment equations become 
the Euler equations of gas dynamics, which can be viewed as a mean-field model near 
equilibrium for the mesoscopic single-particle kinetic equation ED. which is itself a limit 
of microscopic equations for each particle [5511142] . 

Taking a larger moment space M one may also get the Navier-Stokes equation as a 
limit [93] . and this hydrodynamic limit can even be justified rigorously under certain 
assumptions m ■ This clearly shows that moment closure methods can link physical 
theories at different scales. 


3.3 Microscopic Closures 

Since there arc limit connections between the microscopic level and macroscopic moment 
equations, it seems plausible that starting from an individual-based network model, one 
may motivate moment closure techniques. Here we shall illustrate this approach for the 
SIS-model from Section [2. 3 1 Suppose we start at the level of first-order moments and let 
M = {mi,ms}- To close (I2ll) - (I551) we want a map 

m S i = H(mi,m s ). (43) 

If we view the density of the I nodes and S nodes as very weakly correlated random 
variables then a first guess is to use the approximation 

m S i = (SI) ~ (S)(I) = m s mi. 

Plugging (1551) into (l2ll) - (l22l) yields the mean-field SIS model 

m' s = jmi — rmsmi , 
m} = rmsmi — to /. 


(44) 

(45) 
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The mean-field SIS model is one of the simplest examples where one clearly sees that 
although the moment equations are linear ODEs, the moment-closure ODEs are fre¬ 
quently nonlinear. It is important to note that (I44[) is not expected to be valid for all 
possible networks as it ignores the graph structure. A natural alternative is to consider 


m S i = (SI) « m d (S')(/) = m d TO S m/, (46) 

where m d is the mean degree of the given graph/network. Hence it is intuitive that (PHI) 
is valid for a complete graph in the limit K —> oo m- 

If we want to find a closure similar to the approximation (1441) for second-order mo¬ 
ments with M. as in (EH1) . then the classical choice is the pair-approximation (5411751 [751 


TFlab^Flbc 

TYlabc ~ 

m b 


a,b,c£ {5, 1} 


(47) 


which just means that the density of triplet motifs is given approximately by counting 
certain link densities that form the triplet. In (1471) we have again ignored pre-factors 
from the graph structure such as the mean excess degree mm- As before, the assump¬ 
tion S3) is neglecting certain correlations and provides a mapping 


(mss/, m/s/) = H(m n , m SS , m S i ) = ( mssmsi , msimsi \ (48) 

v ms ms ) 

and substituting (1751) into (E3l) - (E5l) yields a system of five closed nonlinear ODEs. Many 
other paradigms for similar closures exist. The idea is to use the interpretation of the 
moments and approximate certain higher-order moments based upon certain assump¬ 
tions for each moment/motif. In the cases discussed here, this means neglecting certain 
correlation terms from random variables. At least on a formal level, this is approach is 
related to the other closures we have discussed. For example, forcing maximum entropy 
means minimizing correlations in the system while assuming a certain distribution for 
the moments just means assuming a particular correlation structure of mixed moments. 


3.4 Geometric Closure 

All the moment closure methods described so far, have been extensively tested in many 
practical examples and frequently lead to very good results; see Section [4] However, 
regarding the question (Q2) on approximation accuracy of moment closure, no completely 
general results are available. To make progress in this direction I conjecture that a high- 
potential direction is to consider moment closures in the context of geometric invariant 
manifold theory. There is very little mathematically rigorous work in this direction [143] 
although the relevance [3T1I1 16] is almost obvious. 

Consider the abstract moment equations (1291) . Let us assume for illustration purposes 
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that we know that (1291) can be written as a system 


dmi 

"ar 

= /ii(toi,to 2 , ... 


dm 2 
d t 

= h 2 (mi,m 2 ,... 

5 TTIki • • •)> 

dm K 
d t 

= h K (m 1 ,m 2 ,... 

1 VTIki ^'/■v+1 5 2l • • •) • 

d m „ +1 

c\t. 

= i/i K+ i(?ni,?n 2 ,. 

.., m K , • ■ ■ 

dm K+ 2 
d t 

= i/i K+2 (TOi,m 2 ,. 

• • ? 1? 2) • • ■ 


where 0 < e -C 1 is a small parameter and each of the component functions of the vector 
field h is of order 0(1) as e —> 0. Then (PE)1) is a fast-slow system |7IJ[S8] with fast 
variables (m K +i,m K + 2 , ...) and slow variables (mi,... , m K ). The classical quasi-steady- 
state assumption [132] to reduce (l49l) to a lower-dimensional system is to take 

0=%±i, 0 = %S, .... (50) 

at at 

This generates a system of differential-algebraic equations and if we can solve the alge¬ 
braic equations 


0 = /i re+ i(mi,m 2 ,...), 0 =/i K+ 2 (mi,m 2 ,...), ■■■ (51) 

via a mapping H as in (13(1 we end up with a closed system of the form (IXO . 

The quasi-steady-state approach hides several difficulties that are best understood 
geometrically from the theory of normally hyperbolic invariant manifolds, which is well 
exemplified by the case of fast-slow systems. For fast-slow systems, the algebraic equa¬ 
tions m provide a representation of the critical manifold 

C 0 = {(mi, m 2 , hj = 0 for j > n, j G N}. (52) 

However, it is crucial to note that, despite its name, Co is not necessarily a manifold but 
in general just an algebraic variety. Even if we assume that Co is a manifold and we would 
be able to find a mapping H of the form (1301) . this mapping is generically only possible 
locally [401188] . Even if we assume in addition that the mapping is possible globally, then 
the dynamics on Co given by (1501) does not necessarily approximate the dynamics of the 
full moment system for e > 0. The relevant property to have a dynamical approximation 
is normal hyperbolicity , i.e., the ’matrix’ 

j, l G {k -f-1, k + 2,...} (53) 

has no eigenvalues with zero real parts; in fact, this matrix is just the total derivative 
of the fast variables restricted to points on Cq but for moment equations it is usually 
infinite-dimensional. Even if we assume in addition that Co is normally hyperbolic, 
which is a very strong and non-generic assumption for a fast-slow system [ 7X1,188] . then 
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the dynamics given via the map H is only the lowest-order approximation. The correct 
full dynamics is given on a slow manifold 

C e = {(m K+1 ,m K+ 2 , ■■■) = H(m 1 ,m 2 ,. ■. ,m K ) + O(e)} (54) 

so H is only correct up to order 0(e). This novel viewpoint on moment closure shows why 
it is probably quite difficult |60j to answer the approximation question (Q2) since for a 
general nonlinear system, the moment equations will only admit a closure via an explicit 
formula locally in the phase space of moments. One has to be very lucky, and probably 
make very effective use of special structures [4611117] in the dynamical system, to obtain 
any global closure. Local closures are also an interesting direction to pursue DU- 


4 Applications &; Further References 

Historically, applications of moment closure can at least be traced back to the classi¬ 
cal Kirkwood closure [75] as well as statistical physics applications, e.g., in the Ising 
model The Gaussian (or normal) closure has a long history as well [155] . In me¬ 
chanical applications and related nonlinear vibrations questions, stochastic mechanics 
models have been among the first where moment closure techniques for stochastic pro¬ 
cesses have become standard tools naM] including the idea to just discard higher-order 
moments [12S] . By now, moment closure methods have permeated practically all natural 
sciences as evidenced by the classical books [21 1151] . For SDEs, moment closure methods 
have not been used as intensively as one may guess but see m- 

For kinetic theory, closure methods also have a long history, particularly starting 
from the famous Grad 13- moment closure [5211146] . and moment methods have become 
fundamental tools in gas dynamics [145] . One particularly important application for 
kinetic-theory moment methods is the modelling of plasmas [5911127] . In general, it is 
quite difficult to study the resulting kinetic moment equations analytically mm but 
many numerical approaches exist [55 [ I55 []1~02II149[ . Of course, the maximum entropy 
closure we have discussed is not restricted to kinetic theory m and maximum entropy 
principles appear in many contexts [11 11511541 [5511 1 24] . 

One area where moment closure methods are employed a lot recently is mathematical 
biology. For example, the pair approximation m and its variants nsi are frequently 
used in various models including lattice models [36ll4Tl!42^lT01. 108lll31j . homogeneous 
networks [11811130] and many other network models [5lll22ll26ll54j . Several closures have 
also included higher-order moments [67P74] and truncation ideas are still used [1511161161] . 
Applications to various different setups for epidemic spreading are myriad [6T1162] . A 
typical benchmark problem for moment methods in biology is the stochastic logistic 
equation [51 15511T001111 011 11 b l 13 . 1 39' . Furthermore, spatial models in epidemiology and 
ecology have been a focus [9lH98lHl4 fll5j. There are several survey and comparison 
papers with a focus on epidemics application and closure-methods available [171H041HU71 
m- There is also a link from mathematical biology and moment closure to transport 
and kinetic equations [53[[51], e.g., in applications of cell motion [55]. Also physical 
constraints, as we have discussed for abstract kinetic equations, play a key role in biology, 
e.g., trying to guarantee non-negativity [62] . 
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Another direction is network dynamics m, where moment closure methods have 
been used very effectively are adaptive, or co-evolutionary, networks with dynamics of 
and on the network [531154] . Moment equations are one reason why one may hope 
to describe self-organization of adaptive networks [201 by low-dimensional dynamical 
systems models [561 . Applications include opinion formation [7811109 j with a focus on 
the classical voter model I120II141II152] ; see [29| for a review of closure methods applied to 
the voter model. Other applications are found again in epidemiology [55ll87l[97ll 133111 341 
[I48]1156| and in game theory [2B1E3H1S5] - The maximum entropy-closure we introduced 
for kinetic equations has also been applied in the context of complex networks [1281 and 
spatial network models in biology [l2l| . An overview of the use of the pair approximation, 
several models, and the relation to master equations can be found in [49]. It has also been 
shown that in many cases low-order or mean-field closures can still be quite effective [50]. 

On the level of moment equations in network science, one has to distinguish between 
purely moment or motif-based choices of the space M and the recent proposal to use 
heterogeneous degree-based moments. For example, instead of just tracking the moment 
of a node density, one also characterizes the degree distribution [48] of the node via new 
moment variables [34] . Various applications of heterogeneous moment equations have 
been investigated [9811135] , 

Another important applications are stochastic reaction networks [3HESD], where the 
mean-field reaction-rate equations are not accurate enough [99]. A detailed computation 
of moment equations from the master equation of reaction-rate models is given in [38] . In 
a related area, turbulent combustion models are investigated using moment closure m 
181 ll OBlIl12ll 29] . For turbulent combustion, one frequently considers so-called conditional 
moment closures where one either conditions upon the flow being turbulent or restricts 
moments to certain parts of phase space; see [823 f° r a ver y detailed review. 

Further applications we have not focused on here can be found in genetics [4] > client- 
server models in computer science [57][58], mathematical finance [1381 . systems biol¬ 
ogy [47], estimating transport coefficients [25], neutron transport ED, and radiative 
transport problems [4311144 . We have also not focused on certain methods to derive 
moment equations including moment-generating functions [98;J 10311153] . Lie-algebraic 
methods [55] , and factorial moment expansions Ca¬ 
in summary, it is clear that many different areas are actively using moment closure 
methods and that a cross-disciplinary approach could yield new insights on the validity 
regimes of various methods. Furthermore, it is important to emphasize again that only 
a relatively small snapshot of the current literature has been given in this review and a 
detailed account of all applications of moment closure methods would probably fill many 
books. 
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